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(54) IMAGE INPUT DEVICE AND IMAGE INPUT METHOD 

(57)Abstract: 

PROBLEM TO BE SOLVED: To allow a photographer to 
easily confirm photographing conditions by informing the 
photographer of whether or not photographing is proper in 
the case of compositing panorama images. 
SOLUTION: A 1st image photographed by an image pickup 
means 2 is used for a criterion image and a 2nd image that 
is being photographed by the image pickup means 2 at 
present is used for a reference image. An overlap area 
detection means 13 detects an overlapped area between 
the criterion image and the reference image and displays 
the size of the overlapped area on an overlapped area 
display indicator of a finder 1 1 . When a shutter of the image 
pickup means 2 is depressed, the overlap area detection 
means 13 lights a warning indicator of the finder 11 when 

the displayed size of the overlapped area is a preset threshold value or below to instruct revision 
of a photographing position to the photographer. 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect 
the original precisely. 

2 **** s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l] The picture input device characterized by to have a lap field detection means 
to by_which a criteria image which picturized previously with an image pick-up means, 
and was memorized for an image storage means in a picture input device which has an 
image pick-up means input an image of a photographic subject, and an image storage 
means memorize an image obtained from an image pick-up means, and an image 
pick-up means detect automatically a lap field with an image under current input. 
[Claim 2] The above-mentioned lap field detection means is a picture input device 
according to claim 1 which detects a lap field with a criteria image from a motion vector 
in a dynamic image under current input. 

[Claim 3] The above-mentioned lap field detection means is a picture input device 
according to claim 1 which laps from a motion vector between a criteria image and an 
image which it is going to picturize, and detects a field. 

[Claim 4] It is the picture input device which is in a picture input device characterized 
by providing the following, and is characterized by a criteria image and an image 
pick-up means detecting a lap field with an image under current input from attitude 
information which searched for the above-mentioned lap field detection means from a 
posture detection sensor signal. An image pick-up means to input an image of a 
photographic subject A posture detection sensor which acquires a signal showing a 
posture of an image pick-up means A lap field detection means by which a criteria 
image which picturized previously with an image storage means and an image pick-up 
means of memorizing an image obtained from an image pick-up means, and was 
memorized for an image storage means, and an image pick-up means detect 
automatically a lap field with an image under current input 

[Claim 5] An image input method which memorizes an image previously picturized with 
an image pick-up means as a criteria image, and is characterized by detecting and 
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expressing a lap field with an image under current input as a criteria image and an 
image pick-up means which were memorized. 

[Claim 6] An image input method according to claim 5 that the above-mentioned criteria 
image and an image pick-up means detect a lap field with an image under current input 
from a motion vector in a dynamic image under current input with an image pick-up 
means. 

[Claim 7] An image input method according to claim 5 that the above-mentioned criteria 
image and an image pick-up means detect a lap field with an image under current input 
from a motion vector between a criteria image and an image which it is going to 
picturize. 

[Claim 8] An image input method according to claim 5 that the above-mentioned criteria 
image and an image pick-up means detect a lap field with an image under current input 
from attitude information of an image pick-up means. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[The technical field to which invention belongs] This invention relates to the picture 
input device and the image input method of compounding the static image of two or 
more sheets photoed with image pick-up means, such as a digital still camera. 
[0002] 

[Description of the Prior Art] The camera which carries out photo electric conversion of 
the image which passed optical system by a CCD sensor etc., carries out A/D conversion 
of the acquired picture signal, and is saved at external storage, such as a memory card, 
has spread. Since the camera which used this CCD sensor etc. can treat the picturized 
image as a digital signal, it has the advantage that processing, processing, and 
transmission of an image can be performed easily. Sticking the image which photoed 
two or more same photographic subjects with such a camera, and compounding the 
panorama image of high resolution is performed. In order to compound this panorama 
image, as some two or more images overlap or two or more images of each are connected 
without a joint, it is necessary to photo a photographic subject. 

[0003] For example, it enables it to check whether when displaying the image which 
operated the already inputted image and the image under current input on a curtailed 
schedule, respectively, and was reduced on the portion which the display adjoined and 
obtaining a panorama image, a current image pick-up location is suitable for the 



2/19 



Japanese Publication number : 11-298837 



method shown in JP,5-161050,A. Moreover, the method shown in JP,9-266561,A 
displays some images which arranged in the image under current input and were 
photoed previously according to the migration direction of a camera, and is changing it 
into the condition that both images were connected. For example, when a camera 
rotates clockwise, it copies out on the left end of a display which started by the field 
which was able to determine beforehand a part for the right edge of the image photoed 
previously, and was established in the camera back, and the image of a current input is 
displayed immediately on the right-hand side by side. And he is trying for the 
sequential image compounded when the shutter was pushed, where the image of two 
sheets is connected to save in panorama memory. 
[0004] 

[Problem(s) to be Solved by the Invention] however, the number of sheets of the image 
on which this check is difficult for and sticks the relation between images, and a 
duplication field especially with the image of the low resolution operated on a curtailed 
schedule for a check reason since the image which operated the already inputted image 
and the image under current input on a curtailed schedule by the method shown in 
JP,5-161050,A, respectively, and was reduced is displaying - increasing - many - in 
case the image of several sheets divides, it becomes impossible to be disregarded this 
problem Moreover, by the method shown in JP,9-266561,A, since a panorama image 
cannot be obtained unless the image of two sheets is in the condition connected 
completely, camera actuation will become the severe thing of a limit. 
[0005] A user is notified of whether when improving this demerit and performing 
various image processings, such as panorama image composition and a three-dimension 
configuration reload of a photographic subject, and composition, photography actuation 
is appropriate for this invention, and it aims at offering the picture input device and the 
image input method of checking photography conditions easily. 
[0006] 

[Means for Solving the Problem] The picture input device concerning this invention is 
characterized by to have a lap field detection means to by_which the criteria image 
which picturized previously with an image pick-up means, and was memorized for an 
image storage means, and an image pick-up means detect automatically a lap field with 
the image under current input in the picture input device which has an image pick-up 
means input an image of a photographic subject, and an image storage means memorize 
an image obtained from an image pick-up means. 

[0007] The above-mentioned lap field detection means is good to detect a lap field with a 
criteria image from a motion vector in a dynamic image under current input. 
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[0008] Moreover, the above-mentioned lap field detection means laps from a motion 
vector between a criteria image and an image which it is going to picturize, and may 
detect a field. 

[0009] An image pick-up means by which other picture input devices concerning this 
invention input an image of a photographic subject, A posture detection sensor which 
acquires a signal showing a posture of an image pick-up means, It is in a picture input 
device which has a lap field detection means by which a criteria image which picturized 
previously with an image storage means and an image pick-up means of memorizing an 
image obtained from an image pick-up means, and was memorized for an image storage 
means, and an image pick-up means detect automatically a lap field with an image 
under current input. The above-mentioned lap field detection means is characterized by 
a criteria image and an image pick-up means detecting a lap field with an image under 
current input from attitude information searched for from a posture detection sensor 
signal. 

[00 10] An image input method concerning this invention memorizes an image 
previously picturized with an image pick-up means as a criteria image, and is 
characterized by for a criteria image and an image pick-up means which were 
memorized detecting a lap field with an image under current input, and compounding 
an image. 

[00 11] It is good to detect a lap field with an image under current input from a motion 
vector between images which are going to detect from a motion vector in a dynamic 
image under current input with an image pick-up means, or it is going to picturize with 
a criteria image, and to detect it from attitude information of **** and an image pick-up 
means with the above-mentioned criteria image and an image pick-up means. 
[0012] 

[Embodiment of the Invention] The picture input device of this invention has the image 
pick-up means and the mode setting means of having ****** sensing elements, such as 
CCD, a main control means, a signal-processing means, a frame memory, external 
storage, a finder, and a lap field detection means. 

[0013] If multiple radiography mode is chosen with the mode setting means of this 
picture input device, a main control means displays the indicator in which it is shown 
that multiple radiography mode was chosen into the finder, it will lap, will notify that 
multiple radiography mode was set up to a field detection means, and will start a lap 
field detection means. If one photographic subject image is photoed with an image 
pick-up means in this condition, a main control means will be memorized to a frame 
memory by using the picturized image of the 1st sheet as a criteria image, and will be 
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memorized also to external storage. If it carries out after photography, and regularity 
minute time amount progress is carried out, shifting the location of an image pick-up 
means to photo a photographic subject, a main control means will be incorporated to a 
frame memory with an image pick-up means by using the image of the 2nd sheet under 
current input as a reference image. If the image under current input is captured by the 
frame memory as a reference image, a lap field detection means will detect the lap field 
of the criteria image memorized to the frame memory which is the image of the 1st 
sheet, and the reference image under current input, and will display lap area size on the 
lap field display indicator of a finder. This processing is repeated for every fixed time 
amount until the shutter of an image pick-up means is depressed. If the shutter of an 
image pick-up means is depressed, it will lap and a lap field detection means will check 
whether it is beyond the displayed threshold as which area size is beforehand set up and 
is displayed with the indicator of a finder. When lap area size is below a threshold as a 
result of this check, an alarm indicator is made to turn on and modification of a camera 
station is required of a photography person. Thus, when compounding two or more 
images, a proper lap field can be chosen. 

[0014] And if a photography person changes a camera station, lap area size with the 
reference image which is carrying out the current input with the criteria image becomes 
beyond a threshold and the shutter of an image pick-up means is depressed, the image 
which is carrying out the current input will be photoed and it will replace by the image 
of the 2nd sheet which photoed the criteria image which memorized the photoed image 
of the 2nd sheet to external storage, and was memorized to the frame memory. This 
processing is wound and ****(ed) while continuing photography, and two or more 
images are compounded in a proper lap field. 
[0015] 

[Example] Drawing 1 is the block diagram showing the configuration of the picture 
input device of one example of this invention. A picture input device 1 has the image 
pick-up means 2, the mode setting means 3, the main control means 4, the 
signal-processing means 5, the memory control means 6, a frame memory 7, the 
enternal memory interface 8, external storage 9, the display interface 10, a finder 11, 
and the external display 12 and the lap field detection means 13 that consists of liquid 
crystal installed in the case of a picture input device 1. 

[0016] The image pick-up means 2 has ****** sensing elements, such as CCD, on the 
optical path which condensed in the lens unit, and outputs the analog video signal 
showing a photographic subject image. The mode setting means 3 sets up the 
photography mode of a picture input device 1, and is installed in the case surface of a 
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picture input device 1, for example, can choose the photography mode of a request of a 
user now. ON / off change in multiple radiography mode is also carried out to this mode 
setting means 3. The main control means 4 directs that the set point in the photography 
mode from the mode setting means 3 changes the contents of processing of reception, 
the signal-processing means 5, the memory control means 6, and lap field detection 
means 13 grade while controlling fundamental processing of photography, a display, 
record, etc. After the signal-processing means 5 pretreats amplification, a clamp, etc. 
and carries out A/D conversion to the analog video signal outputted from the image 
pick-up means 2, it performs various image processings, such as decomposition of 
filtering and a color component. The memory control means 6 reads the picture signal 
which stored in the frame memory 7 the picture signal processed with the 
signal-processing means 5, and was stored in the frame memory 7. A frame memory 7 
consists of VRAM, SRAM, a DRAM, etc., and the image inputted by carrying out a 
current image pick-up with the image photoed last time with the image pick-up means 2 
is accumulated. When saving the picture signal stored in the frame memory 7 at 
external storage 9, the picture signal stored in the frame memory 7 by the memory 
control means 6 is read, and after the signal-processing means 5 performs signal 
processing, such as picture compression, to the picture signal by which reading 
appearance was carried out, it is saved through the enternal memory interface 8 at 
external storage 9. Although IC memory card, a magneto-optic disk, etc. are used as this 
external storage 9, a picture signal may be transmitted to the record medium of a direct 
remote place via a network using a modem card or an ISDN card. Moreover, when 
reading the picture signal saved at external storage 9, the picture signal saved at 
external storage 9 is sent to the signal-processing means 5 through the enternal 
memory interface 8. The signal-processing means 5 performs image elongation of the 
sent picture signal. The picture signal by which reading appearance was carried out 
from this external storage 9 and frame memory 7 indicates by delivery through the 
display interface 10 at a finder 11 or the external display 12, after performing signal 
processing, such as D/A conversion and amplification, with the signal-processing means 
5. The lap field detection means 12 laps with the lap area size of the image inputted by 
carrying out a current image pick-up with the image and the image pick-up means 2 
which were photoed last time which was memorized to the frame memory 7, and detects 
the location of a field. 

[0017] The picture input device 1 constituted as mentioned above explains actuation 
when duplication mode is chosen from the mode setting means 3 with reference to the 
flow chart of drawing 2 . First, if multiple radiography mode is chosen with the mode 
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setting means 3, as shown in drawing 3 , the main control means 22 displays the 
indicator 111 in which it is shown that multiple radiography mode was chosen into the 
finder 11, will lap and will notify that multiple radiography mode was set up to the field 
detection means 13 (step Si). It laps by the notice in this multiple radiography mode, 
and the field detection means 20 is started. If one photographic subject image is photoed 
with the image pick-up means 2 in this condition, the main control means 4 will be 
memorized to a frame memory 7 and external storage 9 by using picturized image 21a of 
the 1st sheet as a criteria image, as shown in drawing 4 (step S2). If it carries out after 
photography, and regularity minute time amount progress is carried out, shifting the 
location of an image pick-up means 2 to photo a photographic subject, the main control 
means 4 will be incorporated to a frame memory 7 with the image pick-up means 2 by 
using image 21b of the 2nd sheet under current input as a reference image (step S3). If 
image 21b under current input is incorporated by the frame memory 7 as a reference 
image, the lap field detection means 13 will detect the lap field of the criteria image 
memorized to the frame memory 7 which is the image of the 1st sheet, and the reference 
image under current input, and will display lap area size on the lap field display 
indicator 112 of a finder 11 (step S4). This processing is repeated for every fixed time 
amount until the shutter of the image pick-up means 2 is depressed (step S5). If the 
shutter of the image pick-up means 2 is depressed, it will lap and the lap field detection 
means 13 will check whether it is beyond the displayed threshold as which area size is 
beforehand set up and is displayed with the indicator 113 of a finder 11 (step S6). When 
lap area size is below a threshold as a result of this check, an alarm indicator 114 is 
made to turn on and modification of a camera station is required of a photography 
person (step S7). And the image which is carrying out the current input if a 
photography person changes a camera station, lap area size with the reference image 
which is carrying out the current input with the criteria image becomes beyond a 
threshold and the shutter of the image pick-up means 2 is depressed is photoed. It 
replaces by image 21b of the 2nd sheet which photoed the criteria image which 
memorized photoed image 21b of the 2nd sheet to external storage 9, and was 
memorized to the frame memory 7 (step S8, S9). Then, photography processing of an 
image of the 3rd sheet is started (steps S10 and S3). This processing is repeated while 
continuing photography. 

[0018] Thus, when photoing a photographic subject continuously and carrying out 
panorama image composition, a photography person can check the propriety of the 
magnitude of duplication between the images to photo during the period from the time 
of the last shutter actuation to the following shutter depression. 
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[0019] Although the above-mentioned example explained the case where the image 
photoed last time was used as a criteria image, it may detect a lap by using as a criteria 
image the image of the arbitration photoed the image photoed to the 1st sheet, or until 
now. Moreover, although the case where inputted a reference image for every fixed time 
interval, lapped, and a field was detected was explained, when half-push [ a shutter ], 
for example, just before photoing the following image, a reference image is inputted, it 
laps, and you may make it detect a field. Furthermore, it is not necessary to set up 
image pick-up mode in the picture input device which specialized in image composition 
or the three-dimension configuration reload of a photographic subject. 
[0020] Moreover, although the above-mentioned example explained the case where the 
lap field detection means 13 detected the lap field of the image under present input, and 
the image photoed last time, it can also ask for a lap field with the reference image 
inputted from the motion vector of the dynamic image under input with the image 
pick-up means 2 a criteria image and now. In this case, for the lap field detection means 
13, as shown in the block diagram of drawing 5 , it has the focus setting means 131, the 
motion vector calculation means 132, and the addition means 133. And the image 
photoed last time is accumulated as a criteria image, and is put on a frame memory 7, 
and the newest picture signal of two frames which follows the image pick-up means 2 in 
time among the images by which the current input is carried out is accumulated in a 
frame memory 7. Here, the newest image of two frames which continues in time among 
the images by which the current input is carried out is called an image (t* 1) and Image t 
from the young order of time amount, respectively. This image (t* l) and Image t are 
updated for every fixed time interval. 

[0021] In an image (t-l), as shown in drawing 6 , the focus setting means 131 
determines the location of the focus xiO and yiO on the screen beforehand, extracts the 
shade pattern of (2P+1 (2N+1)) centering on Focus Pi (xiO, yiO), and creates the 
correlation aperture 30. The motion vector calculation means 132 computes the motion 
vector on the image between an image (t- 1) and Image t. For example, the motion vector 
between an image (t- 1) and Image t is performed by searching for the point (henceforth 
corresponding points) which is mostly in agreement with the shade pattern of the 
correlation aperture 30 created by the image (t"l) in Image t by the correlation 
operation. An example which detects corresponding points with the MACHINGU block 
by the correlation operation here is explained. As shown in drawing 7 , when matching 
the i-th focus Pi (xiO, yiO) in an image (t-l), and the point Qi (xiO+dxi, yiO+dyi) in Image 
t by block matching using the correlation apertures 30 and 31 of (2N+1) and (2P+1), the 
cross-correlation value Si is calculated by following the (l) formula. 
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[0022] 

[Equation l] 

[0023] (l) Concentration [ in / on a formula and / in It'l (x y) / the point (x y) of an image 
(t-l) ], It (x y) centers the concentration in the point (x y) of Image t, and It-1 (x y) bar on 
the point (x y) of an image (t-l) (2N+1). The average concentration in the correlation 
aperture 30 of (2P+l) and It (x y) bar are the average concentration in the correlation 
aperture 31 of (2P+1 (2N+1)) centering on the point (x y) of Image t. 
[0024] By asking for the corresponding points Qi (xiO+dxi, yiO+dyi) which are beyond 
the thresholds as which the maximum of the cross-correlation value Si was beforehand 
determined to each focus Pi one by one, motion vector vi= (dxi, dyi) in Point Pi is called 
for. Moreover, when the maximum of the cross-correlation value Si is below a threshold, 
corresponding points presuppose that it does not exist. Motion vector vt-1 of the whole 
and t are called for by equalizing the motion vector vt of each feature top-most vertices 
Pi, as shown for example, in following the (2) type. The addition means 133 computes 
motion vector v to the image photoed next from a criteria image by adding motion vector 
vt-1 and t which the motion vector calculation means 132 computed, as shown in 
following the (3) type. 
[0025] 

[Equation 2] 

[0026] If motion vector v in this time is called for, it will lap with the indicator 112 in a 
finder 11, and area size will be displayed. 

[0027] The picture input device 1 constituted as mentioned above explains actuation 
when duplication mode is chosen from the mode setting means 3 with reference to the 
flow chart of drawing 8 . If multiple radiography mode is chosen with the mode setting 
means 3, the main control means 22 displays the indicator 1 1 1 in which it is shown that 
multiple radiography mode was chosen into the finder 11, it will lap, will notify that 
multiple radiography mode was set up to the field detection means 13, and will start the 
lap field detection means 20 (step Sll). If one photographic subject image is photoed 
with the image pick-up means 2 in this condition, the main control means 4 will be 
memorized to a frame memory 7 and external storage 9 by using the picturized image of 
the 1st sheet as a criteria image. When memorizing to a frame memory 7 by using this 
image of the 1st sheet as a criteria image, time of day which photoed this criteria image 
is set to t= 1, and motion vector v from a criteria image is initialized to a zero vector 
(step S12). After photoing the image of the 1st sheet, if the location of an image pick-up 
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means 2 to photo a photographic subject is moved, the main control means 4 will 
incorporate the dynamic image which is moving under present input with the image 
pick-up means 2 to a frame memory 7 for every fixed time amount. When incorporating 
a dynamic image to this frame memory 7, it incorporates to a frame memory 7 with the 
image pick-up means 2 by using the image of two sheets under current input as a 
reference image, setting that time of day to t, and using as (t-l) time of day which 
captured the image in front of one (step S13). The focus setting means 131 of the heavy 
field detection means 13 sets up two or more focus in the image photoed at the time of 
day in front of one (t-l), and creates the correlation aperture 30 (step S14). The motion 
vector calculation means 132 asks for motion vector vt-1 between the reference image of 
time of day (t-l), and the reference image of time of day t, and t, it laps by adding 
serially motion vector vt- 1 and t for which it asked, detects a field, and displays lap area 
size on the lap field display indicator 112 of a finder 11 (step S15). This processing is 
repeated for every fixed time amount until the shutter of the image pick-up means 2 is 
depressed (step S16). If the shutter of the image pick-up means 2 is depressed, it will 
lap and the lap field detection means 13 will check whether it is beyond the displayed 
threshold as which area size is beforehand set up and is displayed with the indicator 
113 of a finder 11 (step S17). When lap area size is below a threshold as a result of this 
check, an alarm indicator 1 14 is made to turn on and modification of a camera station is 
required of a photography person (step S18). And if a photography person changes a 
camera station, lap area size with the reference image which is carrying out the current 
input with the criteria image becomes beyond a threshold and the shutter of the image 
pick-up means 2 is depressed, the image which is carrying out the current input will be 
photoed and it will replace by the image of the 2nd sheet which photoed the criteria 
image which memorized the photoed image of the 2nd sheet to external storage 9, and 
was memorized to the frame memory 7 (steps S19 and S20). Then, photography 
processing of an image of the 3rd sheet is started (steps S21 and S3). This processing is 
repeated while continuing photography. 

[0028] Although each above-mentioned example explained the case where incorporated 
the dynamic image inputted into the image pick-up means 2 for every fixed time 
amount, and a lap field with a criteria image was detected as a reference image after 
photoing the image of the 1st sheet when duplication mode was chosen from the mode 
setting means 3 It laps from the motion vector between the images and criteria images 
which half-, and are going to photo it, and may be made to carry out direct detection of 
the field. [ images ] [ the shutter of the image pick-up means 2 ] Thus, if the lap field of 
the image and criteria image which half-and are going to photo it is detected, a 
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photography person can check the lap area size between the images which it is going to 
photo at any time during the period from the time of shutter half push initiation to the 
following shutter depression. [ an image ] [ a shutter ] In this case, for the lap field 
detection means 13, as shown in drawing 9 , it has the focus setting means 131 and the 
motion vector calculation means 132. In the criteria image which is an image photoed 
before one, as shown in drawing 6 , the focus setting means 131 extracts the shade 
pattern of (2P+1 (2N+1)) centering on Focus Pi (xiO, yiO), and creates the correlation 
aperture 30. The motion vector calculation means 132 computes the motion vector 
between the reference image which is an image under current input, and a criteria 
image. 

[0029] The picture input device 1 constituted as mentioned above explains actuation 
when duplication mode is chosen from the mode setting means 3 with reference to the 
flow chart of drawing 10 . If multiple radiography mode is chosen with the mode setting 
means 3, the main control means 22 displays the indicator 111 in which it is shown that 
multiple radiography mode was chosen into the finder 11, it will lap, will notify that 
multiple radiography mode was set up to the field detection means 13, and will start the 
lap field detection means 20 (step S3l). If one photographic subject image is photoed 
with the image pick-up means 2 in this condition, the main control means 4 will be 
memorized to a frame memory 7 and external storage 9 by using the picturized image of 
the 1st sheet as a criteria image (step S32). The focus setting means 131 of the lap field 
detection means 13 sets up two or more focus in the criteria image memorized to the 
frame memory 7, and creates a correlation aperture (step S33). After photoing the 
image of the 1st sheet, if the shutter of the image pick-up means 2 is made half-push, 
moving the location of an image pick-up means 2 to photo a photographic subject, the 
main control means 4 will be incorporated to a frame memory 7 by using as a reference 
image the image which is moving under current input with the image pick-up means 2 
(step S34). The motion vector calculation means 132 of the lap field detection means 13 
computes the motion vector between a reference image and a criteria image, laps, 
detects a field, and displays lap area size on the lap field display indicator 112 of a 
finder 11 (step S35). This processing is repeated for every fixed time amount until the 
shutter of the image pick-up means 2 is depressed (steps S36 and S34). If the shutter of 
the image pick-up means 2 is depressed, the lap field detection means 13 will check 
whether it is beyond the threshold as which the lap area size of the reference image at 
that time and a criteria image is beforehand set up, and is displayed with the indicator 
113 of a finder 11 (steps S36 and S37). When lap area size is below a threshold as a 
result of this check, an alarm indicator 114 is made to turn on and modification of a 



11/19 



Japanese Publication number : 11-298837 



camera station is required of a photography person (step S38). And a photography 
person changes a camera station, the image which is carrying out the current input if 
lap area size with the reference image which is carrying out the current input with the 
criteria image becomes beyond a threshold is photoed, and it replaces by the image of 
the 2nd sheet which photoed the criteria image which memorized the photoed image of 
the 2nd sheet to external storage 9, and was memorized to the frame memory 7 (steps 
S36, S37-S40). Then, photography processing of an image of the 3rd sheet is started 
(steps S41 and S33). This processing is repeated while continuing photography. 
[0030] Moreover, although the above-mentioned example explained the case where it 
asked for a lap field with the reference image which is carrying out the current input 
with the criteria image from the motion vector of the image under input with the image 
pick-up means 2, a lap field with the reference image which is carrying out the current 
input with the criteria image from the posture under photography of the image pick-up 
means 2 may be detected. 

[0031] In this case, in a picture input device 1, as shown in the block diagram of drawing 
11 , it has the posture detection sensor 14 formed in the image pick-up means 2 other 
than the image pick-up means 2, the mode setting means 3, the main control means 4, 
the signal-processing means 5, the memory control means 6, a frame memory 7, the 
enternal memory interface 8, the enternal memory means 9, the display interface 10, a 
finder 11, the external display means 12, and the lap field detection means 13. The lap 
field detection means 13 has the posture calculation means 134, the relative posture 
calculation means 135, the motion vector calculation means 136, and buffer memory 
137, as shown in the block diagram of drawing 12 . From the signal from the posture 
detection sensor 14, the posture calculation means 134 computes the posture of the 
image pick-up means 2 at the time of photography, and saves it at buffer memory 137. 
The relative posture calculation means 135 computes both relative posture from the 
posture at the time of the last photography, i.e., criteria image photography, saved at 
buffer memory 137, and the posture at the time of this reference image input outputted 
from the posture calculation means 134. The motion vector calculation means 136 
computes the motion vector between both images from the relative posture of a criteria 
image and a reference image. 

[0032] The posture detection sensor 14 consists of a 3 shaft acceleration sensor which 
detects the gravitational acceleration of 3 shaft acceleration sensor which detects the 
gravitational acceleration of 3 shaft orientations which intersect perpendicularly 
mutually, for example, 3 shaft magnetometric sensor which detects the earth 
magnetism of 3 shaft orientations, or 3 shaft orientations which intersect 
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perpendicularly mutually, and a 3 axial-angle speed sensor which detects the angular 
velocity of the circumference of 3 shafts. Thus, the posture can be detected, when the 
posture detection sensor 14 is constituted from a 3 shaft acceleration sensor and a 3 
shaft magnetometric sensor and it moves with the speed which is the degree to which a 
quiescent state or acceleration can disregard the image pick-up means 2. Moreover, the 
posture can be detected, when the posture detection sensor 14 is constituted from a 3 
shaft acceleration sensor and a 3 axial-angle speed sensor and it moves with the speed 
which is the degree which can detect the angular velocity of the image pick-up means 2. 
[0033] The posture detection sensor 14 is constituted from a 3 shaft acceleration sensor 
and a 3 shaft magnetometric sensor here, and the actuation in the case of detecting the 
lap field of a criteria image and a reference image is explained. First, as are shown in 
drawing 13 , and make the positive direction of the z-axis of the xyz system of 
coordinates of the image pick-up means 2 into the direction of an optical axis, the 
positive direction of a x axis is made into the right sense direction to the image side 22, 
the positive direction of the y-axis is made into the direction of facing down to the image 
side 22 and it is shown in drawing 14 The positive direction of the Y-axis of a world 
coordinate is made into the sense of gravitational acceleration, make the positive 
direction of the Z-axis into the magnetic sense, and let the positive direction of the 
X-axis be the sense which makes a right-hand orthogonal set in the order of XYZ. 
Moreover, since it is easy, the motion velocity produced by migration of the image 
pick-up means 2 can be disregarded, gravitational acceleration and a magnetic field 
intersect perpendicularly, and it is assumed that a magnetic field does not exist other 
than earth magnetism. In addition, although the magnetic dip of earth magnetism 
exists strictly and earth magnetism and gravitational acceleration do not intersect 
perpendicularly, if a magnetic dip is known, it is calculable like the case where the 
sense and gravitational acceleration of earth magnetism intersect perpendicularly. 
Moreover, if three shafts detect earth magnetism, even when a magnetic dip is strange, 
it is computable in attitude information. That is, the positive direction of the Z-axis is 
taken for the positive direction of the X-axis to the north sense for the east sense. And 
the posture detection sensor 14 is installed so that the x axis of gravitational 
acceleration, the y-axis, a z-axis component, and the x axis of earth magnetism, the 
y-axis and a z-axis component may be detected. 

[0034] The posture signal which this posture detection sensor 14 outputs is sent to the 
lap field detection means 13, after signal processing and A/D conversion, such as 
filtering and amplification, are performed in the signal-processing means 5. The posture 
calculation means 137 of the lap field detection means 13 computes the posture of the 
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image pick-up means 2 from the sent posture signal. (4) and the rotation matrix of 
following the (5) type on the basis of a world coordinate describe the posture of the 
image pick-up means 2. 
[0035] 

[Equation 3] 

[0036] Now, the gravitational acceleration vector g and the earth magnetism vector M 
are expressed with following the (6) type in a world coordinate, respectively, and 
acceleration vector a and the earth magnetism vector m on the basis of a 
device-coordinate system which were detected by 3 shaft acceleration sensor and 3 shaft 
magnetometric sensor are expressed with following the (7) type, respectively. 
[0037] 

[Equation 4] 

[0038] The relation between this gravitational acceleration vector g, the earth 
magnetism vector M, and acceleration vector a and the earth magnetism vector m is 
described by following the (8) type and (9) types using the rotation matrix R. 
[0039] 

[Equation 5] 

[0040] The angle of rotation alpha of the circumference of the X-axis and the angle of 
rotation gamma of the circumference of the Z-axis are calculated by following the (10) 
type and (11) formulas, and the angle of rotation beta of the circumference of a Y-axis is 
calculated by following the (12) type and (13) formulas from the earth magnetism vector 
m from this (8) type using the angle of rotation alpha of the circumference of the X-axis 
for which it asked, and the angle of rotation gamma of the circumference of the Z-axis. 
[0041] 

[Equation 6] 

[0042] Thus, the angle of rotation alpha of the circumference of the X-axis, the angle of 
rotation beta of the circumference of a Y-axis, the angle of rotation gamma of the 
circumference of the Z-axis, and the rotation matrix R can be computed from the 
detection value of 3 shaft acceleration sensor and 3 shaft magnetometric sensor, and the 
posture of the image pick-up means 2 against a world coordinate can be described. 
[0043] The relative posture calculation means 135 computes both relative posture from 
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the rotation matrix Rs over the world coordinate in a criteria image, and the rotation 
matrix Rr over the world coordinate in a reference image. For example, the rotation 
matrix from image pick-up system of coordinates to the image pick-up system of 
coordinates of a reference image of a criteria image is expressed with following the (14) 
type on the basis of a criteria image. 
[0044] 

[Equation 7] 

[0045] From the rotation matrix computed as mentioned above, the motion vector 
calculation means 136 computes the motion vector from a criteria image to a reference 
image, when it assumes that the optical system of the image pick-up means 2 is a 
central projection model which sets a focal distance to f as shown in drawing 13 , the 
look vector ps over the point on a criteria image (x y) is boiled and expressed with 
following the (15) type. 
[0046] 

[Equation 8] 

[0047] The photographic subject image by which the distance of the image pick-up 
means 2 and a photographic subject is expressed with ps of (15) types in a criteria image 
here when sufficiently large compared with the camera station change between a 
criteria image and a reference image is projected on the coordinate of following the (16) 
type in a reference image. 
[0048] 

[Equation 9] 

[0049] However, k is a coefficient for setting z component of the look vector pr to f. 
Therefore, it can ask for motion vector v from a criteria image to a reference image by 
following the (17) type. 
[0050] 

[Equation 10] 

[0051] I is a unit matrix here. (17) Although motion vector v of a formula changes with 
the coordinates (x y) in a criteria image, if the angle of visibility of the image pick-up 
means 2 is not so large, motion vector v hardly receives effect in a coordinate (x y). for 
example, (x y), a photograph center - namely, (0 0) - ****** - it can substitute for (17) 
types and can ask for motion vector v by following the (18) formula. 
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[0052] 

[Equation 11] 

[0053] The picture input device 1 constituted as mentioned above explains actuation 
when duplication mode is chosen from the mode setting means 3 with reference to the 
flow chart of drawing 15 . If multiple radiography mode is chosen with the mode setting 
means 3, the main control means 22 displays the indicator 111 in which it is shown that 
multiple radiography mode was chosen into the finder 11, it will lap, will notify that 
multiple radiography mode was set up to the field detection means 13, and will start the 
lap field detection means 20 (step S5l). If one photographic subject image is photoed 
with the image pick-up means 2 in this condition, the main control means 4 will be 
memorized to a frame memory 7 and external storage 9 by using the picturized image of 
the 1st sheet as a criteria image. Moreover, the posture signal of the image pick-up 
means 2 detected by the posture detection sensor 14 laps through the signal-processing 
means 5, it is sent to the field detection means 13, and the posture of the image pick-up 
means 2 when photoing the image of the 1st sheet computed with the posture 
calculation means 134 is memorized by buffer memory 137 (step S52). When having 
shifted the location of an image pick-up means 2 to photo a photographic subject after 
photoing the image of the 1st sheet The main control means 4 is incorporated to a frame 
memory 7 with the image pick-up means 2 for every fixed time amount by using the 
image of the 2nd sheet under present input as a reference image. The posture of the 
image pick-up means 2 when lapping the posture signal of the image pick-up means 2 
detected by the posture detection sensor 14 through the signal-processing means 5, and 
photoing the image of the 2nd sheet with delivery and the posture calculation means 
134 for the field detection means 13 is computed (step S53). the posture at the time of 
the criteria image photography in which the relative posture calculation means 135 of 
the heavy field detection means 13 was saved at buffer memory 137, and a posture with 
both more relative than the posture at the time of this reference image input outputted 
from the posture calculation means 134 " computing - the motion vector calculation 
means 136 ■- sending . The motion vector calculation means 136 computes and adds 
motion vector v between both images from the relative posture of a criteria image and a 
reference image, detects a lap field, and displays lap area size on the lap field display 
indicator 112 of a finder 11 (step S54). This processing is repeated for every fixed time 
amount until the shutter of the image pick-up means 2 is depressed (step S55). If the 
shutter of the image pick-up means 2 is depressed, it will lap and the lap field detection 
means 13 will check whether it is beyond the displayed threshold as which area size is 
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beforehand set up and is displayed with the indicator 113 of a finder 11 (step S56). 
When lap area size is below a threshold as a result of this check, an alarm indicator 114 
is made to turn on and modification of a camera station is required of a photography 
person (step S57). And if a photography person changes a camera station, lap area size 
with the reference image which is carrying out the current input with the criteria image 
becomes beyond a threshold and the shutter of the image pick-up means 2 is depressed, 
the image which is carrying out the current input will be photoed and it will replace by 
the image of the 2nd sheet which photoed the criteria image which memorized the 
photoed image of the 2nd sheet to external storage 9, and was memorized to the frame 
memory 7 (steps S58 and S59). Then, photography processing of an image of the 3rd 
sheet is started (steps S60 and S53). This processing is repeated while continuing 
photography. Thus, a photography person can check the lap area size between the 
photoed image and the image inputted now during the period of the bottom of the 
following shutter from the time of the last shutter actuation. 

[0054] Although the case where the above-mentioned example computed a motion 
vector from the attitude information of the image pick-up means 2, lapped, and a field 
was detected was explained As shown in drawine 16 , for example, among image side 
23b of image side 23a of a criteria image, and a reference image If the angle of visibility 
of a focal distance or the image pick-up means 2 is known when it is detected that 
rotation of an angle theta arose in the circumference of the yaxis of a device -coordinate 
system, the lap field 24 is easily detectable from the angle of visibility of an angle theta, 
a focal distance, or the image pick-up means 2. Thus, it laps and a field may be detected. 
[0055] 

[Effect of the Invention] This invention uses as a criteria image the image previously 
picturized with the image pick-up means, as explained above. Since a lap field with the 
image under current input is detected automatically and displayed with the criteria 
image and the image pick-up means, when performing the various image processings 
and composition of panorama image composition, a three-dimension configuration 
reload of a photographic subject, etc. A photography person can be notified of whether 
photography actuation is appropriate, and two or more images can be photoed in a 
suitable lap field. 

[0056] Moreover, since the range with which an image laps is not forced beforehand, a 
photography person can photo a photographic subject on comparatively free conditions. 
[0057] Moreover, the lap field of two or more images can improve [ detection ****** ] 
precision with a criteria image and an image pick-up means by detecting a lap field with 
the image under current input from the motion vector in the dynamic image under 
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current input with an image pick-up means. 

[0058] Furthermore, the lap field of two or more images by easy processing can improve 
[ detection ****** ] precision by detecting from the motion vector between a criteria 
image and the image which is going to picturize the lap field with the image under 
present input with the criteria image with the image pick-up means. Moreover, since 
the time of criteria image photography to the time of the next image photography does 
not need to catch a photographic subject, a photography person can operate an image 
pick-up means comparatively freely. 

[0059] Moreover, since a picture signal is not used while a lap field is easily detectable 
by detecting a lap field with the image under present input from the posture of an image 
pick-up means with a criteria image and an image pick-up means, even when a 
photographic subject with few features, such as a texture, is photoed, it stabilizes and 
laps and a field can be detected. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing ll It is the block diagram showing the configuration of the picture input 
device of the example of this invention. 

[Drawing 2] It is the flow chart which shows actuation of the above-mentioned example. 
[Drawing 31 It is the block diagram showing the indicator of a finder. 
[Drawing 41 It is explanatory drawing showing the lap condition of an image. 
[Drawing 5] It is the block diagram showing the configuration of the lap field detection 
means of the 2nd example. 

[Drawing 6l It is explanatory drawing showing the correlation aperture centering on the 
focus. 

[Drawing 7] It is explanatory drawing showing matching of the focus. 
[Drawing 8] It is the flow chart which shows actuation of the 2nd example. 
[Drawing 9l It is the block diagram showing the configuration of the lap field detection 
means of the 3rd example. 

[Drawing 10] It is the flow chart which shows actuation of the 3rd example. 
[Drawing 111 It is the block diagram showing the configuration of the picture input 
device of the 4th example. 

[Drawing 121 It is the block diagram showing the configuration of the lap field detection 
means of the 4th example. 

[Drawing 131 It is explanatory drawing showing the device -coordinate system of an 
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image pick-up means. 

[Drawing 141 It is explanatory drawing showing a world coordinate. 
[Drawing 15] It is the flow chart which shows actuation of the 4th example. 
[Drawing 161 It is the plot plan showing the example of the lap of the image side of a 
criteria image, and the image side of a reference image. 
[Description of Notations] 

1 Picture Input Device 

2 Image Pick-up Means 

3 Mode Setting Means 

4 Main Control Means 

5 Signal-Processing Means 

6 Memory Control Means 

7 Frame Memory 

8 Enternal Memory Interface 

9 External Storage 

10 Display Interface 

11 Finder 

12 External Display 

13 Lap Field Detection Means 

14 Posture Detection Sensor 

131 Focus Setting Means 

132 Motion Vector Calculation Means 

133 Addition Means 

134 Posture Calculation Means 

135 Relative Posture Calculation Means 

136 Motion Vector Calculation Means 

137 Buffer Memory 
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